Image-Based Flexible Endoscope Steering

Manually steering the tip of a flexible endoscope to navigate through an endoluminal path relies on the physician’s dexterity and experience. In this paper we present the realization of a robotic flexible endoscope steering system that uses the endoscopic images to control the tip orientation towards the direction of the lumen. Two image-based control algorithms are investigated, one is based on the optical flow and the other is based on the image intensity. Both are evaluated using simulations in which the endoscope was steered through the lumen. The RMS distance to the lumen center was less than 25% of the lumen width. An experimental setup was built using a standard flexible endoscope, and the image-based control algorithms were used to actuate the wheels of the endoscope for tip steering. Experiments were conducted in an anatomical model to simulate gastroscopy. The image intensity- based algorithm was capable of steering the endoscope tip through an endoluminal path from the mouth to the duodenum accurately. Compared to manual control, the robotically steered endoscope performed 68% better in terms of keeping the lumen centered in the image

Mechatronic design of the Twente humanoid head

This paper describes the mechatronic design of the Twente humanoid head, which has been realized in the purpose of having a research platform for human-machine interaction. The design features a fast, four degree of freedom neck, with long range of motion, and a vision system with three degrees of freedom, mimicking the eyes. To achieve fast target tracking, two degrees of freedom in the neck are combined in a differential drive, resulting in a low moving mass and the possibility to use powerful actuators. The performance of the neck has been optimized by minimizing backlash in the mechanisms, and using gravity compensation. The vision system is based on a saliency algorithm that uses the camera images to determine where the humanoid head should look at, i.e. the focus of attention computed according to biological studies. The motion control algorithm receives, as input, the output of the vision algorithm and controls the humanoid head to focus on and follow the target point. The control architecture exploits the redundancy of the system to show human-like motions while looking at a target. The head has a translucent plastic cover, onto which an internal LED system projects the mouth and the eyebrows, realizing human-like facial expressions

Endoscopic Camera Control by Head Movements for Thoracic Surgery

In current video-assisted thoracic surgery, the endoscopic camera is operated by an assistant of the surgeon, which has several disadvantages. This paper describes a system which enables the surgeon to control the endoscopic camera without the help of an assistant. The system is controlled using head movements, so the surgeon can use his/her hands to oper- ate the instruments. The system is based on a flexible endoscope, which leaves more space for the surgeon to operate his/her instruments compared to a rigid endoscope. The endoscopic image is shown either on a monitor or by means of a head- mounted display. Several trial sessions were performed with an anatomical model. Results indicate that the developed concept may provide a solution to some of the problems currently encountered in video-assisted thoracic surgery. The use of a head-mounted display turned out to be a valuable addition since it ensures the image is always in front of the surgeon’s eyes

Bridging the gap between passivity and transparency

In this paper a structure will be given which in a remarkably simple way offers a solution to the implementation of different telemanipulation schemes for discrete time varying delays by preserving passivity and allowing the highest trans- parency possible. This is achieved by splitting the communication channel in two separate ones, one for the energy balance which will ensure passivity and one for the haptic information between master and slave and which will address transparency. The authors believe that this structure is the most general up to date which preserves passivity under discrete time varying delays allowing different control schemes to address transparency

Image-based robotic steering of advanced flexible endoscopes and instruments

Flexible endoscopy allows the physician to examine the internal body cavities of the patient in a minimally invasive way. Advanced flexible endoscopes and instruments are being developed, which will enable the physician to perform in- terventions that are not possible using conventional endoscopes. However, these endoscopes and instruments are difficult to use, because they are not ergonomic, their control is not intuitive, and multiple physicians are required to work to- gether to perform the procedure. In order to allow a single physician to con- trol the advanced flexible endoscope and the instruments in an intuitive way, a robotic solution is envisioned, in which the physician controls all degrees of free- dom from a surgical console. In this thesis, several aspects of the robotic steering of the endoscope and the instruments are investigated. For the endoscope, steering the tip with a haptic device is evaluated. In this study, novices and experienced endoscopists steer the endoscope to perform a simulated colonoscopy. Haptic feedback is provided to help the subject to steer the endoscope towards the lumen. The lumen position is detected from the en- doscopic images using image processing. This steering method was compared to conventional endoscope steering, and to steering without haptic feedback. The results show that using a haptic device may be a viable alternative method for the steering of advanced flexible endoscopes. The results suggest that the use of haptic cues may reduce patient discomfort. For the steering of the instruments, hysteresis that is present in the system is a major issue. This is caused by friction, compliance, and free play. The system parameters are in general unknown, since they change during the procedure. Thus, online estimation of the system parameters is desired in order to reduce the hysteresis effect. This estimation requires knowing the position of the tip of the endoscopic instrument. However, adding sensors to measure the tip position is difficult, since the space at the tip is very limited and because of sterilization issues. Therefore, estimation of the tip position from the endoscopic images is proposed. This is realized using a virtual visual servoing approach. A model of the instrument is updated to match the actual instrument that is observed in the endoscopic images. Two methods are compared: with and without adding visual markers to the endoscopic instrument. The two methods perform similarly, and are able to estimate the position of the tip with an RMS error of less than 1.8mm in the horizontal, vertical, and away-from-camera directions. The developed tip position estimation algorithm is used to improve the con- trol of the endoscopic instruments. A hysteresis estimation and compensation system is proposed which uses the estimated instrument tip position to reduce the hysteresis that is present. In an experimental validation, it is shown that the proposed system can reduce the hysteresis by approximately 75% for all degrees of freedom of the instrument. Finally, tele-operated steering of the hysteresis-compensated instrument is evaluated. The method is compared to the manual control handle that was originally used to steer the instrument. Subjects performed a tapping task us- ing both methods. The results show a reduction of the average task completion time by 67% when using the tele-operated steering. The results from these studies show that steering an advanced flexible endo- scope and its instruments from a surgical console is viable. This would enable a single physician to perform interventions using a flexible endoscope that are currently not yet possible

Image-based hysteresis reduction for the control of flexible endoscopic instruments

The limited dexterity of conventional flexible endoscopic instruments restricts the clinical procedures that can be performed by flexible endoscopy. Advanced instruments with a higher degree of dexterity are being developed, but are difficult to control manually. Adding actuators to these instruments may make them easier to control. However, the intrinsic hysteresis that is present between the actuators and the tip of the instrument needs to be reduced in order to allow accurate control. We present an esti- mation algorithm that determines the hysteresis between the actuators and the instrument tip in all three degrees of freedom of the instrument: insertion, rotation, and bending. The estimation is performed on-line. The endoscopic images are used as the only feedback, and no additional sensors are placed on the instrument, which is beneficial for application in clinical practice. The estimated parameters are used to reduce the hysteresis that is present. Experimental validation showed a hysteresis reduction of 75%, 78%, and 73% for the insertion, rotation, and bending degrees of freedom, respectively

Pose reconstruction of flexible instruments from endoscopic images using markers

A system is developed that can reconstruct the pose of flexible endoscopic instruments that are used in ad- vanced flexible endoscopes using solely the endoscopic images. Four markers are placed on the instrument, whose positions are measured in the image. These measurements are compared to a three-dimensional rendered model of the instrument. The pseudo-inverse of the interaction matrix between the state of the model and the marker positions in the image is used to update the state such that the model will track the real instrument. An experiment was performed in which the instrument was moved inside a colon model, while the tip position was simultaneously measured with an electromagnetic tracking system. The root mean square errors of the position estimation were 2.3 mm, 2.2 mm and 1.7 mm in the horizontal (x), vertical (y) and away-from-camera (z) directions, respectively

3D position estimation of flexible instruments: marker-less and marker-based methods

Purpose Endoscopic images can be used to allow accurate flexible endoscopic instrument control. This can be implemented using a pose estimation algorithm, which estimates the actual instrument pose from the endoscopic images. Methods In this paper, two pose estimation algorithms are compared: a marker-less and a marker-based method. The marker-based method uses the positions of three markers in the endoscopic image to update the state of a kinematic model of the endoscopic instrument. The marker-less method works similarly, but uses the positions of three feature points instead of the positions of markers. The algorithms are evaluated inside a colon model. The endoscopic instrument is manually operated, while an X-ray imager is used to obtain a ground-truth reference position. Results The marker-less method achieves an RMS error of 1.5, 1.6, and 1.8 mm in the horizontal, vertical, and away-from-camera directions, respectively. The marker-based method achieves an RMS error of 1.1, 1.7, and 1.5 mm in the horizontal, vertical, and away-from-camera directions, respectively. The differences between the two methods are not found to be statistically significant. Conclusions The proposed algorithms are suitable to realize accurate robotic control of flexible endoscopic instruments, enabling the physician to perform advanced procedures in an intuitive way